Differential Logging: A Commutative and Associative Logging Scheme for Highly Parallel Main Memory Databases

With a gigabyte of memory priced at less than $2,000, the main-memory DBMS (MMDBMS) is emerging as an economically viable alternative to the disk-resident DBMS (DRDBMS) in many problem domains. The MMDBMS can show significantly higher performance than the DRDBMS by reducing disk accesses to the sequential form of log writing and the occasional checkpointing. Upon the system crash, the recovery process begins by accessing the diskresident log and checkpoint data to restore a consistent state. With the increasing CPU speed, however, such disk access is still the dominant bottleneck in the MMDBMS. To overcome this bottleneck, this paper explores altematives of parallel logging and recoveiy. The major contribution of this paper is the so-called d$ferential logging scheme that permits unrestricted parallelism in logging and recovery. Using the bit-wise X O R operation both to compute the differential log between the before and after images and to recover the consistent database state, this scheme offers the mom for significant performance improvement in the MMDBMS. First, with logging done on the difference, the log volume is reduced to almost half compared with the conventional physical logging. Second, the commutativiq and associativity of XOR enables processing of log records in an arbitrary order: This means that we can freely distribute log records to multiple disks to improve the logging pelforniance. During the recovery time, we can do parallel restart independently f o r each log disk. This paper shows the superior performance of the differential logging comparatively with the physical logging in the shared-memory multiprocessor environment.